Notifying apparatus, image capturing apparatus, notifying method, image capturing method, and storage medium

ABSTRACT

There is provided a notifying apparatus. A detecting unit detects a motion amount of an object from an image obtained through first shooting, the first shooting being carried out repeatedly at predetermined intervals of time. A converting unit converts the motion amount into a motion blur amount that will arise in second shooting, on the basis of the predetermined intervals of time and an exposure time used in the second shooting. A notifying unit makes a notification of motion blur on the basis of the motion blur amount. The notifying unit changes a form of the notification in accordance with a magnitude of the motion blur amount.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a divisional of application Ser. No. 16/728,434,filed Dec. 27, 2019, the entire disclosure of which is herebyincorporated by reference.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to a notifying apparatus, an imagecapturing apparatus, a notifying method, an image capturing method, anda storage medium.

Description of the Related Art

Recently, some commercial models of image capturing apparatuses such asdigital still cameras are provided with a shooting mode called “shutterspeed priority mode”. The shutter speed priority mode is a shooting modein which the photographer sets a desired shutter speed, and the imagecapturing apparatus then automatically sets exposure setting valuesaside from the shutter speed, such as the aperture value, ISOsensitivity, and so on. The photographer can shoot an image at his orher preferred shutter speed by using the shutter speed priority mode inthis manner. For example, setting a fast shutter speed before shootingan image and then shooting the image in the shutter speed priority modemakes it possible to shoot an image having little motion blur.

Japanese Patent Laid-Open No. 2008-172667 discloses a technique thatmakes it possible for a photographer to visually confirm a motion regionduring preparatory shooting. “Preparatory shooting” refers to shootingcarried out by the photographer to compose the shot, set the shootingconditions, and so on while looking at an electronic viewfinder or rearsurface LCD of the image capturing apparatus. According to the techniquedisclosed in Japanese Patent Laid-Open No. 2008-172667, a motion regionis detected between time-series images capturing during preparatoryshooting, and that motion region is displayed in an emphasized manner.

As described above, to shoot an image with little motion blur, it isnecessary to shoot the image using a fast shutter speed. However, evenif a fast shutter speed is set during preparatory shooting and theactual shooting is carried out using the shutter speed priority mode,there are situations where an object will be captured in a blurry state.

For example, when shooting a footrace runner in the shutter speedpriority mode so that little motion blur arises, the photographerestimates the speed at which the runner is moving during the preparatoryshooting, and sets a shutter speed he or she thinks will result inlittle motion blur for the runner. However, it is difficult for thephotographer to confirm whether or not motion blur is arising at the setshutter speed, even if he or she visually confirms the image displayedin an electronic viewfinder or rear surface LCD during preparatoryshooting. Specifically, it is difficult for the photographer to visuallyconfirm motion blur in small regions, such as the arms and legs of therunner, during the preparatory shooting. Furthermore, if the shutterspeed differs between the actual shooting and the preparatory shooting,the motion blur will also be different between the actual shooting andthe preparatory shooting, and it is therefore difficult to confirmmotion blur in the actual shooting even when the image has been visuallyconfirmed during the preparatory shooting. Thus when taking an actualshot at the set shutter speed, the runner will be blurred in thecaptured image if the shutter speed is too slow relative to the speed atwhich the runner is moving.

Furthermore, in this conventional technique, it is not easy for thephotographer to know how much motion blur will arise in the actualshooting in advance.

SUMMARY OF THE INVENTION

Having been achieved in light of such circumstances, the presentinvention provides a technique that makes it easy for a photographer toknow how much motion blur will arise in actual shooting in advance.

According to a first aspect of the present invention, there is provideda notifying apparatus comprising at least one processor and/or at leastone circuit which functions as: a detecting unit configured to detect amotion amount of an object from an image obtained through firstshooting, the first shooting being carried out repeatedly atpredetermined intervals of time; a converting unit configured to convertthe motion amount into a motion blur amount that will arise in secondshooting, on the basis of the predetermined intervals of time and anexposure time used in the second shooting; and a notifying unitconfigured to make a notification of motion blur on the basis of themotion blur amount, wherein the notifying unit changes a form of thenotification in accordance with a magnitude of the motion blur amount.

According to a second aspect of the present invention, there is provideda notifying apparatus comprising at least one processor and/or at leastone circuit which functions as: a detecting unit configured to detect,in a predetermined period, a motion amount of an image capturingapparatus carrying out first shooting that is carried out repeatedly atpredetermined intervals of time; a converting unit configured to convertthe motion amount into a motion blur amount that will arise in secondshooting, on the basis of the predetermined period and an exposure timeused in the second shooting; and a notifying unit configured to make anotification of motion blur on the basis of the motion blur amount,wherein the notifying unit changes a form of the notification inaccordance with a magnitude of the motion blur amount.

According to a third aspect of the present invention, there is provideda notifying apparatus comprising at least one processor and/or at leastone circuit which functions as: a detecting unit configured to detect amotion amount of an object from an image obtained through firstshooting, the first shooting being carried out repeatedly atpredetermined intervals of time; a converting unit configured to convertthe motion amount into a motion blur amount that will arise in secondshooting, on the basis of the predetermined intervals of time and anexposure time used in the second shooting; and a notifying unitconfigured to make a notification of motion blur on the basis of themotion blur amount, wherein the notifying unit changes a form of thenotification in accordance with a divergence between the motion bluramount and a target motion blur amount.

According to a fourth aspect of the present invention, there is providedan image capturing apparatus comprising: the notifying apparatusaccording to the first aspect; and an image sensor.

According to a fifth aspect of the present invention, there is providedan image capturing apparatus comprising: the notifying apparatusaccording to the second aspect; and an image sensor.

According to a sixth aspect of the present invention, there is providedan image capturing apparatus comprising: the notifying apparatusaccording to the third aspect; and an image sensor.

According to a seventh aspect of the present invention, there isprovided an image capturing apparatus comprising at least one processorand/or at least one circuit which functions as: an image capturingcontrol unit configured to control an image sensor to carry out firstshooting repeatedly at a predetermined framerate; a detecting unitconfigured to detect a motion amount of an object from an image obtainedthrough the first shooting; a changing unit configured to change thepredetermined framerate on the basis of a difference between the motionamount and a target motion blur amount; a converting unit configured toconvert the motion amount into a motion blur amount that will arise insecond shooting, on the basis of the predetermined framerate and anexposure time used in the second shooting; and a notifying unitconfigured to make a notification of motion blur on the basis of themotion blur amount.

According to an eighth aspect of the present invention, there isprovided a notifying method executed by a notifying apparatus,comprising: detecting a motion amount of an object from an imageobtained through first shooting, the first shooting being carried outrepeatedly at predetermined intervals of time; converting the motionamount into a motion blur amount that will arise in second shooting, onthe basis of the predetermined intervals of time and an exposure timeused in the second shooting; and making a notification of motion blur onthe basis of the motion blur amount, wherein a form of the notificationis changed in accordance with a magnitude of the motion blur amount.

According to a ninth aspect of the present invention, there is provideda notifying method executed by a notifying apparatus, comprising:detecting, in a predetermined period, a motion amount of an imagecapturing apparatus carrying out first shooting that is carried outrepeatedly at predetermined intervals of time; converting the motionamount into a motion blur amount that will arise in second shooting, onthe basis of the predetermined period and an exposure time used in thesecond shooting; and making a notification of motion blur on the basisof the motion blur amount, wherein a form of the notification is changedin accordance with a magnitude of the motion blur amount.

According to a tenth aspect of the present invention, there is provideda notifying method executed by a notifying apparatus, comprising:detecting a motion amount of an object from an image obtained throughfirst shooting, the first shooting being carried out repeatedly atpredetermined intervals of time; converting the motion amount into amotion blur amount that will arise in second shooting, on the basis ofthe predetermined intervals of time and an exposure time used in thesecond shooting; and making a notification of motion blur on the basisof the motion blur amount, wherein a form of the notification is changedin accordance with a divergence between the motion blur amount and atarget motion blur amount.

According to an eleventh aspect of the present invention, there isprovided an image capturing method executed by an image capturingapparatus, comprising: controlling an image sensor to carry out firstshooting repeatedly at a predetermined framerate; detecting a motionamount of an object from an image obtained through the first shooting;changing the predetermined framerate on the basis of a differencebetween the motion amount and a target motion blur amount; convertingthe motion amount into a motion blur amount that will arise in secondshooting, on the basis of the predetermined framerate and an exposuretime used in the second shooting; and making a notification of motionblur on the basis of the motion blur amount.

According to a twelfth aspect of the present invention, there isprovided a non-transitory computer-readable storage medium which storesa program for causing a computer to execute a notifying methodcomprising: detecting a motion amount of an object from an imageobtained through first shooting, the first shooting being carried outrepeatedly at predetermined intervals of time; converting the motionamount into a motion blur amount that will arise in second shooting, onthe basis of the predetermined intervals of time and an exposure timeused in the second shooting; and making a notification of motion blur onthe basis of the motion blur amount, wherein a form of the notificationis changed in accordance with a magnitude of the motion blur amount.

According to a thirteenth aspect of the present invention, there isprovided a non-transitory computer-readable storage medium which storesa program for causing a computer to execute a notifying methodcomprising: detecting, in a predetermined period, a motion amount of animage capturing apparatus carrying out first shooting that is carriedout repeatedly at predetermined intervals of time; converting the motionamount into a motion blur amount that will arise in second shooting, onthe basis of the predetermined period and an exposure time used in thesecond shooting; and making a notification of motion blur on the basisof the motion blur amount, wherein a form of the notification is changedin accordance with a magnitude of the motion blur amount.

According to a fourteenth aspect of the present invention, there isprovided a non-transitory computer-readable storage medium which storesa program for causing a computer to execute a notifying methodcomprising: detecting a motion amount of an object from an imageobtained through first shooting, the first shooting being carried outrepeatedly at predetermined intervals of time; converting the motionamount into a motion blur amount that will arise in second shooting, onthe basis of the predetermined intervals of time and an exposure timeused in the second shooting; and making a notification of motion blur onthe basis of the motion blur amount, wherein a form of the notificationis changed in accordance with a divergence between the motion bluramount and a target motion blur amount.

According to a fifteenth aspect of the present invention, there isprovided a non-transitory computer-readable storage medium which storesa program for causing a computer to execute an image capturing methodcomprising: controlling an image sensor to carry out first shootingrepeatedly at a predetermined framerate; detecting a motion amount of anobject from an image obtained through the first shooting; changing thepredetermined framerate on the basis of a difference between the motionamount and a target motion blur amount; converting the motion amountinto a motion blur amount that will arise in second shooting, on thebasis of the predetermined framerate and an exposure time used in thesecond shooting; and making a notification of motion blur on the basisof the motion blur amount.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating the configuration of an imagecapturing apparatus 100 including a notifying apparatus.

FIG. 2 is a flowchart illustrating a shooting process according to afirst embodiment.

FIG. 3 is a diagram illustrating an example of the configuration of anotification image generating unit 300 included in an image processingunit 107 according to the first embodiment.

FIG. 4 is a flowchart illustrating a process by which the notificationimage generating unit 300 generates a motion blur notification image(step S204 in FIG. 2 ).

FIG. 5A is a diagram illustrating an example of a preparatory shootingimage.

FIG. 5B is a diagram illustrating an example of motion vectors in apreparatory shooting image.

FIG. 6 is a flowchart illustrating a motion vector calculation process(step S402 in FIG. 4 ).

FIG. 7 is a diagram illustrating the motion vector calculation process(step S402 in FIG. 4 ).

FIG. 8 is a diagram illustrating a motion vector in preparatoryshooting, and motion blur, in actual shooting, converted from the motionvector in the preparatory shooting (converted motion blur).

FIGS. 9A to 9E are diagrams illustrating three examples of a motion blurnotification image.

FIG. 10 is a diagram illustrating a histogram of converted motion blurhaving a predetermined value or higher in the scene illustrated in FIG.5A.

FIG. 11 is a block diagram illustrating the configuration of an imagecapturing apparatus 1100 including a notifying apparatus.

FIG. 12 is a diagram illustrating an example of the configuration of anotification image generating unit 1200 included in an image processingunit 1102.

FIG. 13 is a flowchart illustrating a process by which the notificationimage generating unit 1200 generates a motion blur notification image(step S204 in FIG. 2 ).

FIG. 14 is a diagram illustrating an example of the configuration of anotification image generating unit 1400 included in the image processingunit 107 according to a third embodiment.

FIG. 15 is a flowchart illustrating a process by which the notificationimage generating unit 1400 generates a motion blur notification image(step S204 in FIG. 2 ).

FIGS. 16A to 16C are diagrams illustrating converted motion blurcorresponding to the current shutter speed (exposure time) in actualshooting, and converted motion blur when the shutter speed (exposuretime) has been changed.

FIG. 17 is a block diagram illustrating the configuration of an imagecapturing apparatus 1700 including a notifying apparatus.

FIG. 18 is a diagram illustrating an example of the configuration of anotification image generating unit 1800 included in an image processingunit 1701.

FIG. 19 is a flowchart illustrating a process by which the notificationimage generating unit 1800 generates a motion blur notification image(step S204 in FIG. 2 ).

FIG. 20 is a flowchart illustrating a shooting process according to afifth embodiment.

FIG. 21 is a flowchart illustrating a process by which a control unit101 determines shooting conditions for preparatory shooting (step S2008in FIG. 20 ).

FIGS. 22A and 22B are diagrams illustrating a relationship between amotion amount of an object, calculated from a preparatory shootingimage, and a target motion blur (permissible motion amount).

FIGS. 23A and 23B are timing charts illustrating a process by which thecontrol unit 101 determines shooting conditions for preparatory shooting(step S2008 in FIG. 20 ).

FIG. 24 is a diagram illustrating a motion vector in preparatoryshooting, and motion blur, in actual shooting, converted from the motionvector in the preparatory shooting (converted motion blur).

DESCRIPTION OF THE EMBODIMENTS

Hereinafter, embodiments of the present invention will be described withreference to the attached drawings. Elements that are given the samereference numerals throughout all of the attached drawings represent thesame or similar elements, unless otherwise specified. Note that thetechnical scope of the present invention is defined by the claims, andis not limited by the following respective embodiments. Also, not all ofthe combinations of the aspects that are described in the embodimentsare necessarily essential to the present invention. Also, the aspectsthat are described in the individual embodiments can be combined asappropriate.

First Embodiment

FIG. 1 is a block diagram illustrating the configuration of an imagecapturing apparatus 100 including a notifying apparatus. The firstembodiment will describe a configuration that switches a motion blurnotification on and off.

A control unit 101 is a CPU, for example; the control unit 101 reads outcontrol programs for controlling the various blocks of the imagecapturing apparatus 100 from ROM 102 (described later), loads theprograms into RAM 103 (described later), and executes the programs.Through this, the control unit 101 can control the operations of thevarious blocks of the image capturing apparatus 100. The ROM 102 iselectrically erasable/recordable non-volatile memory, and storesparameters and the like necessary for each of the blocks of the imagecapturing apparatus 100 to function, in addition to the control programsfor controlling those blocks. The RAM 103 is rewritable volatile memory,and is used for loading the control programs executed by the controlunit 101 and the like, temporarily storing data generated through theoperations of the blocks of the image capturing apparatus 100, and thelike.

An optical system 104 is constituted by a lens group including a zoomlens and a focus lens, and forms a subject image on an image capturingsurface of an image capturing unit 105, which will be described later.The image capturing unit 105 is an image sensor such as a CCD or a CMOSsensor; the image capturing unit 105 photoelectrically converts anoptical image formed on the image capturing surface of the imagecapturing unit 105 by the optical system 104 and outputs an analog imagesignal obtained as a result to an A/D conversion unit 106. The A/Dconversion unit 106 converts the input analog image signal into digitalimage data. The digital image data output from the A/D conversion unit106 is temporarily stored in the RAM 103.

An image processing unit 107 applies various types of image processing,such as white balance adjustment, color interpolation, and gammaprocessing, to the image data stored in the RAM 103. The imageprocessing unit 107 also includes a notification image generating unit300 (described later), which generates a motion blur notification imageby superimposing an image plane enabling motion blur to be easilyconfirmed over an image stored in the RAM 103.

A recording unit 108 is a removable memory card or the like. Therecording unit 108 records the image data processed by the imageprocessing unit 107 as a recorded image, via the RAM 103. A display unit109 is a display device such as an LCD; the display unit 109 displaysimages stored in the RAM 103, images recorded into the recording unit108, a user interface for operations for accepting instructions from theuser, and so on. The display unit 109 also displays images captured bythe image capturing unit 105, for composing the shot and the like duringpreparatory shooting. An instruction input unit 110 is a touch panel, amouse, or the like. The user inputs instructions to the image capturingapparatus 100 using the instruction input unit 110.

A shooting process executed by the image capturing apparatus 100 will bedescribed next with reference to FIG. 2 . Unless otherwise specified,the processes in the respective steps of this flowchart are realized bythe control unit 101 executing the aforementioned control programs. Theprocessing illustrated in this flowchart starts when the user turns theimage capturing apparatus 100 on and an operating mode of the imagecapturing apparatus 100 enters a shooting mode.

In step S201, the control unit 101 starts preparatory shooting (firstshooting). During the period of the preparatory shooting, the imagecapturing apparatus 100 captures images in sequence, in the same manneras a moving image (a preparatory shooting image), and displays thatimage in the display unit 109. In other words, during the period of thepreparatory shooting, the preparatory shooting is carried out repeatedlyat predetermined intervals of time. The user composes the shot and so onwhile viewing the display preparatory shooting image. Note that theprocesses of steps S202 to S206 (described hereinafter) are carried outduring the period of preparatory shooting.

In step S202, the control unit 101 sets shooting conditions for actualshooting (second shooting) in response to user instructions (useroperations) made using the instruction input unit 110. The shootingconditions include exposure conditions, and the exposure conditionsinclude the shutter speed (exposure time), ISO sensitivity, F-stopvalue, and so on. Note that the shooting conditions for the actualshooting may be automatically set by the control unit 101.

In step S203, the control unit 101 determines whether to turn a motionblur notification on or off. Turning the motion blur notification on oroff can be set by the user using the instruction input unit 110, forexample. When the user sets the motion blur notification on or off, asetting value indicating on or off is held in the RAM 103. The controlunit 101 determines whether the motion blur notification is on or off inaccordance with this setting value. If it has been determined that themotion blur notification is on, the process moves to step S204, and ifnot, the process moves to step S205.

In step S204, under the control of the control unit 101, thenotification image generating unit 300 generates the motion blurnotification image by superimposing a motion blur notification planeonto the preparatory shooting image. In other words, the motion blurnotification image is a preparatory shooting image with the motion blurnotification plane superimposed thereon. The process of step S204 willbe described in detail later with reference to FIG. 4 .

In step S205, the control unit 101 displays an image in the display unit109. Specifically, when the motion blur notification image has beengenerated in step S204 (when it has been determined in step S203 thatthe motion blur notification is on), the control unit 101 displays themotion blur notification image (the preparatory shooting image ontowhich the motion blur notification plane has been superimposed) in thedisplay unit 109. When the motion blur notification image has not beengenerated (when it has been determined in step S203 that the motion blurnotification is off), the control unit 101 displays the preparatoryshooting image (the preparatory shooting image onto which the motionblur notification plane has not been superimposed) in the display unit109.

In step S206, the control unit 101 determines whether or not the userhas pressed a shutter button. The shutter button is included in theinstruction input unit 110, for example. If the shutter button has beenpressed, the process moves to step S207, and if not, the process returnsto step S202.

While viewing the preparatory shooting image or the motion blurnotification image displayed in the display unit 109, the user can pressthe shutter button when he or she has the chance to take a shot. Theuser can easily confirm motion blur during the preparatory shooting whenthe motion blur notification image is being displayed in the displayunit 109. If the motion blur the user has confirmed is not motion blurthat meets his or her preferences, the user can avoid pressing theshutter button, which returns the process to step S202 and makes itpossible to change (reset) the shutter speed (exposure time) for theactual shooting. In this manner, during the preparatory shooting, theuser can repeatedly change the shutter speed (exposure time) for theactual shooting while confirming the motion blur notification imagedisplayed in the display unit 109 until the motion blur that meets hisor her preferences is achieved, and can then press the shutter buttonwhen there is a chance to take a shot.

When the shutter button is pressed in step S206, the control unit 101carries out actual shooting, and records the image from the actualshooting in the recording unit 108, in step S207.

An example of the configuration of the notification image generatingunit 300 included in the image processing unit 107 will be describednext with reference to FIG. 3 . The notification image generating unit300 includes a motion vector calculation unit 301, a converted motionblur calculation unit 302, a motion blur notification plane generationunit 303, and an image superimposing unit 304. The operations of thenotification image generating unit 300 will be described in detail laterwith reference to FIG. 4 .

Next, the process by which the notification image generating unit 300generates the motion blur notification image (step S204 in FIG. 2 ) willbe described in detail with reference to FIG. 4 .

In step S401, the notification image generating unit 300 obtains thepreparatory shooting image captured during the preparatory shooting bythe image capturing apparatus 100. The obtained preparatory shootingimage is input to the motion vector calculation unit 301 and the imagesuperimposing unit 304.

FIG. 5A is a diagram illustrating an example of the preparatory shootingimage. The present embodiment will describe an example in which, asillustrated in FIG. 5A, a scene is shot in which a dog 501 is running tothe left and a dog 502 is standing still.

In step S402, the motion vector calculation unit 301 calculates a motionvector between the preparatory shooting images as motion information. A“motion vector” expresses horizontal and vertical direction movementamounts of an object between preparatory shooting images as a vector.The method for calculating the motion vector will be described in detaillater with reference to FIGS. 6 and 7 .

FIG. 6 is a flowchart illustrating the motion vector calculation processcarried out by the motion vector calculation unit 301 (step S402 in FIG.4 ). Although the present embodiment will describe a block matchingmethod as an example of the method for calculating the motion vector,the method for calculating the motion vector is not limited to thisexample, and may be gradient method instead, for example.

In step S601, the motion vector calculation unit 301 obtains twopreparatory shooting images adjacent with respect to time. The motionvector calculation unit 301 then sets the preparatory shooting imagefrom an Mth frame as a base frame, and sets the preparatory shootingimage from an M+1th frame as a reference frame.

As illustrated in FIG. 7 , in step S602, the motion vector calculationunit 301 arranges a base block 702, made up of N×N pixels, in a baseframe 701.

Also as illustrated in FIG. 7 , in step S603, the motion vectorcalculation unit 301 sets pixels (N+n)×(N+n), which surround coordinates704 that match the center coordinates of the base block 702 in the baseframe 701, in a reference frame 703 as a search range 705.

In step S604, the motion vector calculation unit 301 calculates thecorrelation between the base block 702 in the base frame 701, and anN×N-pixel reference block 706 at coordinates present within the searchrange 705 in the reference frame 703, to calculate a correlation value.The correlation value is calculated on the basis of an inter-framedifference absolute value sum for the pixels in the base block 702 andthe reference blocks 706. In other words, the coordinates where thevalue of the inter-frame difference absolute value sum is lowest are thecoordinates where the correlation value is the highest. Note that themethod for calculating the correlation value is not limited to a methodthat finds the inter-frame difference absolute value sum, and mayinstead be a method for calculating the correlation value on the basisof an inter-frame difference sum of squares, a normal cross-correlationvalue, or the like, for example. The example in FIG. 7 indicates thatthe reference blocks 706 has the highest correlation.

In step S605, the motion vector calculation unit 301 calculates themotion vector on the basis of the reference block coordinates indicatingthe highest correlation value found in step S604. In the example in FIG.7 , the motion vector is found on the basis of the coordinates 704corresponding to the center coordinates of the base block 702 in thebase frame 701, and the center coordinates of the reference block 706,in the search range 705 of the reference frame 703. In other words, theinter-coordinate distance and direction, from the coordinates 704 to thecenter coordinates of the reference block 706, are found as the motionvector.

In step S606, the motion vector calculation unit 301 determines whetheror not a motion vector has been calculated for all of the pixels in thebase frame 701. If the motion vector calculation unit 301 has determinedin step S606 that a motion vector has not been calculated for all of thepixels, the process returns to step S602, whereas if the motion vectorcalculation unit 301 has determined that a motion vector has beencalculated for all of the pixels, the process returns to the flowchartof FIG. 4 .

When the process returns to step S602, the motion vector calculationunit 301 arranges an N×N-pixel base block 702 in the aforementioned baseframe 701, central to a pixel for which a motion vector has not yet beencalculated. The processing from steps S603 to S605 is then carried outin the same manner as described earlier. In other words, the motionvector calculation unit 301 calculates motion vectors for all of thepixels in the base frame 701 by repeating the processing from steps S602to S605 while moving the base block 702 in FIG. 7 .

FIG. 5B illustrates an example of the motion vectors calculated in thismanner. FIG. 5B is a diagram illustrating an example of motion vectorsin the preparatory shooting image indicated in FIG. 5A. The preparatoryshooting image in FIG. 5A is an example in which the dog 501 is runningto the left. FIG. 5B illustrates an example of the motion vectors in thecase where an object is moving in this manner. In the exampleillustrated in FIG. 5B, leftward motion vectors are detected in theregion corresponding to the running dog 501, whereas “0” is detected asthe motion vectors in other regions, such as the dog 502 that isstanding still, the fence in the background, and so on. The motionvectors of “0” are not illustrated.

Note that the motion vector calculation unit 301 may calculate a motionvector every predetermined number of pixels instead of calculatingmotion vectors for all of the pixels.

The motion vector calculation unit 301 calculates the motion vectorsbetween preparatory shooting images adjacent with respect to timethrough the foregoing processing.

Returning to FIG. 4 , in step S403, the converted motion blurcalculation unit 302 obtains the shutter speed (exposure time) for theactual shooting set in step S202 of FIG. 2 , and the time intervalbetween the images in the preparatory shooting, as the shootingconditions.

In step S404, the converted motion blur calculation unit 302 convertsthe motion vectors for each pixel, calculated in step S402, into motionblur in the actual shooting, on the basis of the exposure time for theactual shooting and the time interval between the images in thepreparatory shooting, which were obtained in step S403. The method forconverting the motion vectors from the preparatory shooting into themotion blur in the actual shooting will be described in detail withreference to FIG. 8 .

FIG. 8 is a diagram illustrating a motion vector in preparatoryshooting, and motion blur, in actual shooting, converted from the motionvector in the preparatory shooting (converted motion blur). FIG. 8illustrates an example in which the time interval between the images inthe preparatory shooting is 1/60 seconds, and the exposure time in theactual shooting is 1/120 seconds or 1/30 seconds.

The converted motion blur calculation unit 302 converts the motionvector for each pixel into motion blur in the actual shooting on thebasis of the following conversion equations (1) and (2).

CONV_GAIN=EXP_TIME/INT_TIME  (1)

CONV_BLUR=VEC_LEN×CONV_GAIN  (2)

Here, in Equation (1), CONV_GAIN represents a conversion gain forconverting the motion vector in the preparatory shooting into a motionvector in the actual shooting, EXP_TIME represents the exposure time inthe actual shooting, and INT_TIME represents the time interval betweenimages in the preparatory shooting. In Equation (2), CONV_BLURrepresents the converted motion blur in the actual shooting, and VEC_LENindicates the length of the motion vector in the preparatory shooting.

In Equation (1), the conversion gain CONV_GAIN is calculated by dividingthe exposure time EXP_TIME in the actual shooting by the time intervalINT_TIME between images in the preparatory shooting. In Equation (2),the converted motion blur CONV_BLUR in the actual shooting is calculatedby multiplying the length VEC_LEN of the motion vector by the conversiongain CONV_GAIN.

Specifically, as illustrated in FIG. 8 , when the length VEC_LEN of themotion vector in the preparatory shooting is 10 pixels and the exposuretime EXP_TIME in the actual shooting is 1/120 seconds, the conversiongain CONV_GAIN is ½x, and thus the converted motion blur is 5 pixels.Likewise, when the exposure time EXP_TIME in the actual shooting is 1/30seconds, the conversion gain CONV_GAIN is 2×, and thus the convertedmotion blur is 20 pixels.

Returning to FIG. 4 , in step S405, the motion blur notification planegeneration unit 303 creates an image plane for notifying the user of themotion blur (a motion blur notification plane) on the basis of theconverted motion blur for each pixel, calculated in step S404.

In step S406, the image superimposing unit 304 generates the motion blurnotification image by superimposing the motion blur notification planecreated in step S405 onto the preparatory shooting image.

Three examples of the motion blur notification image will be describedhere with reference to FIG. 9A to 9E. Displaying the motion blurnotification image in the display unit 109 during the preparatoryshooting makes it possible for the user to easily confirm the motionblur.

FIGS. 9A to 9C illustrate an example of notifying the user of the motionblur by displaying an icon. A method for generating the motion blurnotification image by displaying an icon will be described here. In stepS405, of the converted motion blur for each pixel, the motion blurnotification plane generation unit 303 calculates the percentage of thenumber of pixels having a converted motion blur of a predetermined valueor higher, with respect to the entire screen. When the percentage isgreater than or equal to a predetermined percentage, the motion blurnotification plane generation unit 303 determines the converted motionblur, from among the converted motion blur at greater than or equal to apredetermined value, that occupies the greatest percentage in thescreen.

The method for determining the converted motion blur that occupies thegreatest percentage of the screen will be described with reference toFIG. 10 . FIG. 10 is a diagram illustrating a histogram of convertedmotion blur having a predetermined value or higher in the sceneillustrated in FIG. 5A. The motion blur notification plane generationunit 303 can generate a histogram such as that illustrated in FIG. 10 ,and then divide ranges of the converted motion blur on the basis of thegenerated histogram. The number of classes in the histogram can be setin advance. For example, consider a case where peaks (local maximumpoints) are present in the frequency of the histogram, as indicated inFIG. 10 . In this case, the motion blur notification plane generationunit 303 can divide the range of the converted motion blur so that eachdivided range has one peak, with each peak serving as a representativemotion blur for a corresponding part of the object. Note that the peaksin the histogram can be detected using a method similar to that used fordetecting peaks in signal waveforms.

For example, three peaks 1001 to 1003 are present in the frequency ofthe histogram 1000 illustrated in FIG. 10 . As such, the motion blurnotification plane generation unit 303 divides the range of theconverted motion blur at the positions where the frequency of theconverted motion blur first becomes a local minimum, on the lower sidesfrom each peak, and then assigns display colors to each of the resultingdivided ranges. As illustrated in FIG. 10 , when the percentage of theconverted motion blur corresponding to the peak 1003 is the highest, themotion blur notification plane generation unit 303 creates a red motionblur notification icon 901, as indicated in FIG. 9A, as the motion blurnotification plane. Then, in step S406, the image superimposing unit 304generates a motion blur notification image such as that illustrated inFIG. 9A by superimposing the motion blur notification plane, includingthe motion blur notification icon 901, onto the preparatory shootingimage.

When the peak 1002 is higher than the peak 1003 and the percentage ofthe converted motion blur corresponding to the peak 1002 is the highest,the motion blur notification plane generation unit 303 creates a greenmotion blur notification icon 902, as indicated in FIG. 9B, as themotion blur notification plane. Then, in step S406, the imagesuperimposing unit 304 generates a motion blur notification image suchas that illustrated in FIG. 9B by superimposing the motion blurnotification plane, including the motion blur notification icon 902,onto the preparatory shooting image.

Likewise, when the peak 1001 is higher than the peak 1003 and thepercentage of the converted motion blur corresponding to the peak 1001is the highest, the motion blur notification plane generation unit 303creates a blue motion blur notification icon 903, as indicated in FIG.9C, as the motion blur notification plane. Then, in step S406, the imagesuperimposing unit 304 generates a motion blur notification image suchas that illustrated in FIG. 9C by superimposing the motion blurnotification plane, including the motion blur notification icon 903,onto the preparatory shooting image.

Although the foregoing describes dividing the converted motion blurranges using minimum points in the histogram, the converted motion blurranges may be divided at a predetermined threshold instead.Alternatively, a configuration in which the division positions arevaried in accordance with conditions may be employed as well. Even ifsuch a division method is employed, the motion blur notification planegeneration unit 303 can create, as the motion blur notification plane, amotion blur notification icon having a display color corresponding tothe divided range including the converted motion blur that occupies thehighest percentage in the screen.

FIG. 9D illustrates an example of notifying the user of the motion blurby displaying frames. A method for generating the motion blurnotification image by displaying frames will be described here. In stepS405, of the pixels within divided regions of the shooting screen, themotion blur notification plane generation unit 303 calculates thepercentage of the number of pixels having a converted motion blur of apredetermined value or higher, with respect to the overall dividedregions. For divided regions in which that percentage is greater than orequal to a predetermined percentage, the motion blur notification planegeneration unit 303 creates motion blur notification frames, asillustrated in FIG. 9D, as the motion blur notification plane. At thistime, the motion blur notification plane generation unit 303 determinesthe display color of the motion blur notification frames correspondingto each divided region in accordance with the magnitude of the convertedmotion blur in each divided region.

For example, the motion blur notification plane generation unit 303divides the converted motion blur range into three parts on the basis ofthree thresholds, and assigns blue, green, and red to the divided rangesin that order, starting with the lowest converted motion blur. In thiscase, the motion blur notification plane generation unit 303 creates redmotion blur notification frames 904 for the parts corresponding to thelegs, where there is a high amount of converted motion blur, and createsgreen motion blur notification frames 905 for parts corresponding to thehead and tail, where there is a medium amount of converted motion blur.The motion blur notification plane generation unit 303 also creates bluemotion blur notification frames 906 for parts corresponding to thetrunk, where there is little converted motion blur. Then, in step S406,the image superimposing unit 304 generates a motion blur notificationimage such as that illustrated in FIG. 9D by superimposing the motionblur notification plane, including the motion blur notification frames904 to 906, onto the preparatory shooting image.

FIG. 9E illustrates an example of notifying the user of the motion blurby displaying edges in which motion blur has arisen in an enhancedmanner. A method for generating the motion blur notification image bydisplaying the edges in which motion blur has arisen in an enhancedmanner will be described here. In step S405, the motion blurnotification plane generation unit 303 detects the edge strength of thepreparatory shooting image. It is assumed that the edge strength iscalculated using a known technique such as a Sobel filter, so this willnot be described here. The motion blur notification plane generationunit 303 then extracts pixels for which the edge strength is greaterthan or equal to a predetermined value and for which the convertedmotion blur is greater than or equal to a predetermined value. Themotion blur notification plane generation unit 303 then creates a motionblur notification plane, in which the edge areas where motion blur hasarisen are displayed in an enhanced manner for the extracted pixels, asindicated by motion blur notification edges 907 to 909 in FIG. 9E. Atthis time, the motion blur notification plane generation unit 303determines the display color of each pixel in the motion blurnotification edges in accordance with the amount of converted motionblur in those pixels.

For example, the motion blur notification plane generation unit 303divides the converted motion blur range into three parts on the basis ofthree thresholds, and assigns blue, green, and red to the divided rangesin that order, starting with the lowest converted motion blur. In thiscase, the motion blur notification plane generation unit 303 creates redmotion blur notification edges 907 for the pixels in the partscorresponding to the legs, where there is a high amount of convertedmotion blur, and creates green motion blur notification edges 908 forthe pixels in the parts corresponding to the head and tail, where thereis a medium amount of converted motion blur. The motion blurnotification plane generation unit 303 also creates blue motion blurnotification edges 909 for the pixels in the parts corresponding to thetrunk, where there is little converted motion blur. Then, in step S406,the image superimposing unit 304 generates a motion blur notificationimage such as that illustrated in FIG. 9E by superimposing the motionblur notification plane, including the motion blur notification edges907 to 909, onto the preparatory shooting image.

Although FIG. 9E illustrates an example in which the edges are displayedin an enhanced manner by varying the display colors of the motion blurnotification edges in accordance with the amount of converted motionblur, the enhanced display method is not limited thereto. A method ofdisplaying the edges in an enhanced manner by varying the thicknesses ofthe motion blur notification edges in accordance with the amount ofconverted motion blur can be given as another example of an enhanceddisplay method.

According to the first embodiment as described thus far, the imagecapturing apparatus 100 detects the motion amount of an object from animage obtained through preparatory shooting, and converts the motionamount to a motion blur amount arising during actual shooting. The imagecapturing apparatus 100 then notifies the user of the motion blur on thebasis of the motion blur amount. At this time, the image capturingapparatus 100 varies the form of the notification in accordance with theamount of the motion blur. This makes it possible for the photographerto easily know, in advance, how much motion blur will arise in theactual shooting.

The present embodiment describes, with reference to FIGS. 9A to 9E, anexample in which the user is notified of the motion blur when theconverted motion blur is greater than or equal to a predetermined value.However, a configuration in which the user is notified of the motionblur even when the converted motion blur is less than the predeterminedvalue may be employed. This makes it easy for the user to confirmwhether the motion blur is insufficient during the preparatory shootingperiod, such as in long-exposure shooting, where the user wishes to usemotion blur to express a sense of motion.

Additionally, although the present embodiment describes an example inwhich the user is notified of the motion blur when the converted motionblur is greater than or equal to a predetermined value, a configurationmay be employed in which the user is notified having set a range for theconverted motion blur in which a notification is to be made. As anexample of a way for setting the range in which a notification is to bemade, if the user has decided to view the shot image in the display, arange is set in which motion blur can be confirmed visually when theimage is displayed in the designated display at actual size. A method inwhich the range at which motion blur can be recognized in the display isconverted to a range corresponding to the size of the shot image on thebasis of the difference between the number of pixels of the shot imageand the number of pixels of the display can be used as a method forcalculating the range. The image capturing apparatus 100 then notifiesthe user of the converted motion blur when the blur is within theconverted range.

Additionally, although the present embodiment describes an example inwhich the user is notified of the motion blur when the converted motionblur is greater than or equal to a predetermined value, a configurationmay be employed in which the predetermined value is determined on thebasis of information from an angular velocity sensor. When the usertakes a shot while holding the image capturing apparatus 100 in his orher hand, blur resulting from his or her hand shaking, camerawork, andthe like is included as converted motion blur in the calculated range.If a notification is to be made only for the converted motion blur ofthe object, while excluding blur caused by hand shake and camerawork,removing converted motion blur within the same range as the movementsdetected by the angular velocity sensor makes it possible to notify theuser only of motion blur arising in the object.

Additionally, although the present embodiment describes the motion blurnotification icons 901 to 903, the motion blur notification frames 904to 906, and the motion blur notification edges 907 to 909 as threeexamples of the motion blur notification plane, the types of the motionblur notification planes are not limited thereto. For example, aconfiguration may be employed in which all regions where motion blur isarising, including both the edge region and a flat region, are displayedin an enhanced manner. Specifically, the motion blur notification planegeneration unit 303 carries out an enhanced display in which the pixelswhere the per-pixel converted motion blur is greater than or equal to apredetermined value are colored with a color based on the amount ofconverted motion blur. Carrying out the enhanced display for both theedge region and regions aside from the edge region in this manner ensurethe entire object is displayed in an enhanced manner, which makes iteasier to confirm motion blur.

Additionally, although the present embodiment describes a configurationin which the motion blur notification image is displayed in the displayunit 109 as the method for notifying the user of motion blur, the methodfor notifying the user of motion blur is not limited thereto. Forexample, a configuration in which the user is notified of motion blur byoutputting sound may be employed. In this case, for example, the controlunit 101 may output a motion blur notification sound from a speaker (notshown) when, of the converted motion blur for each pixel, the percentageof the number of pixels having a converted motion blur of apredetermined value or higher, with respect to the entire screen, isgreater than or equal to a predetermined percentage. Then, the controlunit 101 may divide the range of the converted motion blur using thesame method as that described with reference to FIG. 10 , assign adifferent volume to each divided range, and then output motion blurnotification sound at a volume corresponding to the divided rangeincluding the converted motion blur that occupies the highest percentagewithin the screen. Alternatively, the control unit 101 may assign adifferent type of sound to each divided range, and output the type ofmotion blur notification sound corresponding to the divided rangeincluding the converted motion blur that occupies the highest percentagein the screen. In other words, the control unit 101 can change at leastone of the type of the sound and the volume in accordance with theamount of converted motion blur.

Additionally, although the present embodiment describes, as a method ofmaking the motion blur notification, an example in which the displaycolor of the motion blur notification icon is changed in accordance withthe amount of converted motion blur, the item to be varied in accordancewith the amount of converted motion blur is not limited to the displaycolor. For example, a configuration may be employed in which the size ofthe motion blur notification icon is varied in accordance with theamount of converted motion blur. For example, a configuration may beemployed in which the icon size is increased when there is moreconverted motion blur, and the icon size is reduced when there is lessconverted motion blur. Varying the icon size in accordance with theamount of converted motion blur in this manner makes it possible for theuser to easily confirm the intensity of the motion blur.

Accordingly, the notification image generating unit 300 can carry outcontrol for displaying blur notification icons, which have differentappearances depending on the amount of converted motion blur, along withthe preparatory shooting image. For example, the notification imagegenerating unit 300 can vary at least one of the color and size of theblur notification icon in accordance with the amount of converted motionblur.

Additionally, although the present embodiment describes, as a method ofmaking the motion blur notification, an example in which the displaycolor of the motion blur notification frames or the motion blurnotification edges is changed in accordance with the amount of convertedmotion blur, the item to be varied in accordance with the amount ofconverted motion blur is not limited to the display color. For example,a configuration may be employed in which the thickness of the lines ofthe motion blur notification frames or the motion blur notificationedges is varied in accordance with the amount of converted motion blur.For example, a configuration may be employed in which the frames oredges are displayed thicker when there is more converted motion blur,and thinner when there is less converted motion blur. Varying the linethickness in accordance with the amount of converted motion blur in thismanner makes it possible for the user to easily confirm the intensity ofthe motion blur.

Accordingly, the notification image generating unit 300 can carry outcontrol for displaying a plurality of blur notification frames, whichsurround a plurality of divided regions in the preparatory shootingimage where motion blur is arising, along with the preparatory shootingimage. The notification image generating unit 300 can then vary theappearance of each blur notification frame in accordance with the amountof converted motion blur in the divided region corresponding to the blurnotification frame. For example, the notification image generating unit300 can vary at least one of the color and size of each blurnotification frame in accordance with the amount of converted motionblur in the divided region corresponding to the blur notification frame.Note that the region where the blur notification frame is displayed isnot limited to the divided region, and a motion blur notification can bemade using a blur notification frame for any desired partial region inthe preparatory shooting image where motion blur is arising.Additionally, the notification image generating unit 300 can display aplurality of edge regions where motion blur is arising in thepreparatory shooting image, in an enhanced manner with different formsdepending on the amount of converted motion blur in each of the edgeregions. The “enhanced display” includes at least one of displaying theplurality of edge regions with different colors depending on the amountof converted motion blur in each of the edge regions, and displaying theplurality of edge regions using lines of different thicknesses dependingon the amount of converted motion blur in each of the edge regions.

The present embodiment describes a configuration in which the motionamount of an object is detected from an image obtained throughpreparatory shooting, and by converting that motion amount to a motionblur amount that will arise in the actual shooting, the motion bluramount that will arise in the actual shooting is estimated. However, themethod for estimating the motion blur amount that will arise in theactual shooting is not limited thereto. Additionally, the two instancesof shooting, i.e., the preparatory shooting and the actual shooting, maybe any two types of shooting (first shooting and second shooting). Forexample, the image capturing apparatus 100 may obtain a first shot imageobtained through the first shooting under first shooting conditions, andmotion information of an object in the first shot image. The motioninformation obtained here is the speed of the object in the first shotimage (e.g., a movement amount in a unit of time, expressed as a numberof pixels). Then, on the basis of the motion information and secondshooting conditions, the image capturing apparatus 100 may estimate themotion blur amount of the object in a second shot image obtained whenthe second shooting is carried out under the second shooting conditions.The second shooting conditions are set independent from the firstshooting conditions. This point also applies to the second to fifthembodiments, which will be described hereinafter.

Second Embodiment

Although the first embodiment described a configuration in which amotion amount detected from a preparatory shooting image is convertedinto a motion blur amount arising in actual shooting, the secondembodiment will describe a configuration in which a motion amount of animage capturing apparatus that is carrying out preparatory shooting isconverted into a motion blur amount arising in actual shooting. Althoughthe second embodiment will be described with reference to an imagecapturing apparatus 1100, illustrated in FIG. 11 , instead of the imagecapturing apparatus 100 illustrated in FIG. 1 , the configuration andoperations of the image capturing apparatus 1100 share some common partswith the image capturing apparatus 100. The following will primarilydescribe points that are different from the first embodiment.

The image capturing apparatus 1100 illustrated in FIG. 11 differs fromthe image capturing apparatus 100 illustrated in FIG. 1 in that anangular velocity detection unit 1101 has been added, and the imageprocessing unit 107 has been replaced with an image processing unit1102.

The angular velocity detection unit 1101 is an angular velocity sensor,for example, which detects the angular velocity of the image capturingapparatus 1100 itself, caused by the user's hand shaking, the user'scamerawork, or the like, in a yaw direction and a pitch direction. Anydesired known method can be used as the method by which the angularvelocity detection unit 1101 detects the angular velocity.

The image processing unit 1102 includes a notification image generatingunit 1200 (described later), which generates a motion blur notificationimage by superimposing an image plane enabling motion blur to be easilyconfirmed over an image stored in the RAM 103.

An example of the configuration of the notification image generatingunit 1200 included in the image processing unit 1102 will be describednext with reference to FIG. 12 . The notification image generating unit1200 illustrated in FIG. 12 differs from the notification imagegenerating unit 300 illustrated in FIG. 3 in that the motion vectorcalculation unit 301 has been replaced with an image motion informationconversion unit 1201. The operations of the notification imagegenerating unit 1200 will be described in detail later with reference toFIG. 13 .

Next, the process by which the notification image generating unit 1200generates the motion blur notification image (step S204 in FIG. 2 ) willbe described in detail with reference to FIG. 13 .

In step S1301, the image motion information conversion unit 1201converts the angular velocity detected by the angular velocity detectionunit 1101 into information corresponding to motion vectors in the image.Equations (3) and (4) indicate approximate transformations forconverting the angular velocity into information corresponding to motionvectors in the image.

$\begin{matrix}{{MOV\_ yaw} \approx \frac{f{\tan\left( {- \frac{\omega\_ yaw}{fps}} \right)}}{pp}} & (3)\end{matrix}$ $\begin{matrix}{{MOV\_ pitch} \approx \frac{f{\tan\left( {- \frac{\omega\_ pitch}{fps}} \right)}}{pp}} & (4)\end{matrix}$

Here, MOV_yaw represents the amount of movement in the yaw direction,and MOV_pitch represents the amount of movement in the pitch direction.f represents the focal length, ω_yaw represents the angular velocity inthe yaw direction, ω_pitch represents the angular velocity in the pitchdirection, fps represents the framerate of the preparatory shooting, andpp represents the pixel pitch of the image capturing unit 105.

According to the approximate transformation equations indicated inEquations (3) and (4), the amount of movement in the image capturingplane is calculated on the basis of the angle of movement in the timeinterval between images in the preparatory shooting, and the focallength; the amount of movement in the image (the number of pixels ofmovement) is then calculated by dividing the amount of movement in theimage capturing plane by the pixel pitch. Here, the calculated amount ofmovement in the image does not differ from pixel to pixel, but isinstead uniform for all of the pixels.

The image motion information conversion unit 1201 treats the amount ofmovement in the yaw direction as an amount of movement in the horizontaldirection and the amount of movement in the pitch direction as an amountof movement in the vertical direction, and then outputs the combinationof these movement amounts to the converted motion blur calculation unit302 as motion vectors uniform for all the pixels.

In step S1302, the motion blur notification plane generation unit 303creates an image plane for notifying the user of the motion blur (themotion blur notification plane) on the basis of the converted motionblur calculated in step S404. The process of step S1302 is the same asin the first embodiment (step S405 in FIG. 4 ) in that the motion blurnotification plane is created with different forms depending on theamount of the converted motion blur. However, in the second embodiment,the motion vectors are uniform for all of the pixels, and thus theconverted motion blur is also uniform for all of the pixels, which meansthat a histogram such as that illustrated in FIG. 10 cannot begenerated. Accordingly, the motion blur notification plane generationunit 303 determines the form of the motion blur notification plane by,for example, comparing the amount of converted motion blur that isuniform for all of the pixels to one or more thresholds. For example,assume that A represents the amount of the converted motion blur, andT1, T2, and T3 represent three thresholds (T1<T2<T3). If T1<A<T2, themotion blur notification plane generation unit 303 creates the bluemotion blur notification icon 903 (FIG. 9C) as the motion blurnotification plane. If T2<A<T3, the motion blur notification planegeneration unit 303 creates the green motion blur notification icon 902(FIG. 9B) as the motion blur notification plane. If T3<A, the motionblur notification plane generation unit 303 creates the red motion blurnotification icon 901 (FIG. 9A) as the motion blur notification plane.

According to the second embodiment as described thus far, the imagecapturing apparatus 1100 converts the motion amount of the imagecapturing apparatus 1100 carrying out preparatory shooting into themotion blur amount that will arise in the actual shooting. This makes itpossible to obtain the converted motion blur with a comparatively lightprocessing load.

The second embodiment describes an example in which the amount ofmovement in the image capturing plane is calculated on the basis of theangle of movement in the time interval between images in the preparatoryshooting, and the focal length, and the amount of movement in the imageis then calculated by dividing the amount of movement in the imagecapturing plane by the pixel pitch. However, the method for calculatingthe amount of movement in the image is not limited thereto. Aconfiguration may be employed in which the amount of movement in theimage capturing plane is calculated on the basis of the angle ofmovement in an exposure period of the image in the preparatory shooting,and the focal length, and the amount of movement in the image is thencalculated by dividing the amount of movement in the image capturingplane by the pixel pitch. Specifically, the angular velocity detectionunit 1101 detects the angular velocity during the exposure period of theimage during the preparatory shooting, and the notification imagegenerating unit 1200 generates the motion blur notification image on thebasis of that angular velocity. Note that in this case, the convertedmotion blur calculation unit 302 converts motion vectors uniform for allthe pixels into motion blur arising in the actual shooting, on the basisof the exposure time for the actual shooting and the exposure time forthe image during the preparatory shooting. In this manner, the imagecapturing apparatus 1100 can calculate the motion blur arising duringactual shooting from the motion amount of the image capturing apparatus1100 (e.g., the angle of movement) in a predetermined period (e.g., thetime interval between images in the preparatory shooting or the exposureperiod of the image in the preparatory shooting).

Third Embodiment

The third embodiment will describe a configuration in which the motionblur notification plane is created with different forms depending ondivergence between the converted motion blur and a target motion blur.In the third embodiment, the basic configuration of the image capturingapparatus 100 is the same as in the first embodiment (see FIG. 1 ). Thefollowing will primarily describe points that are different from thefirst embodiment.

In the present embodiment, the image processing unit 107 of the imagecapturing apparatus 100 includes a notification image generating unit1400 (FIG. 14 ) instead of the notification image generating unit 300(FIG. 3 ). The notification image generating unit 1400 illustrated inFIG. 14 differs from the notification image generating unit 300illustrated in FIG. 3 in that a divergence calculation unit 1401 hasbeen added, and the motion blur notification plane generation unit 303has been replaced with a motion blur notification plane generation unit1402. The operations of the notification image generating unit 1400 willbe described in detail later with reference to FIG. 15 .

Next, the process by which the notification image generating unit 1400generates the motion blur notification image (step S204 in FIG. 2 ) willbe described in detail with reference to FIG. 15 .

In step S1501, the divergence calculation unit 1401 calculates a targetvalue for the converted motion blur in the actual shooting (the targetmotion blur), and finds the divergence between the target motion blurand the converted motion blur for each pixel calculated in step S404.The divergence calculation unit 1401 calculates a converted motion blurthat is permissible within the screen as the target motion blur. Forexample, if the user has decided to view shot images in a display, the“converted motion blur that is permissible within the screen”corresponds to the upper limit of a range at which the motion blurcannot be seen visually when the image is displayed in the designateddisplay at actual size.

The method for calculating the divergence between the converted motionblur and the target motion blur will be described with reference toFIGS. 16A to 16C. FIGS. 16A to 16C are diagrams illustrating convertedmotion blur corresponding to the current shutter speed (exposure time)in actual shooting, and converted motion blur when the shutter speed(exposure time) has been changed. FIGS. 16A to 16C illustrate examplesin which the exposure time in the actual shooting is 1/125 seconds, andthe exposure time after the change is 1/250 seconds, 1/500 seconds, and1/1000 seconds, as the shooting conditions.

The divergence calculation unit 1401 calculates the divergence betweenthe converted motion blur and the target motion blur for each pixel onthe basis of the following Equations (5) to (7).

DIFF=TARG_BLUR/CONV_BLUR  (5)

TARG_TIME=EXP_TIME×DIFF  (6)

STEP=LOG₂(DIFF)  (7)

In Equation (5), TARG_BLUR represents the target motion blur, and DIFFrepresents the difference between the target motion blur and theconverted motion blur (the ratio of the target motion blur to theconverted motion blur, here). In Equation (6), TARG_TIME represents atarget exposure time, and EXP_TIME represents the current exposure time.In Equation (7), STEP represents the number of shutter speed stepsrequired to change from the current exposure time to the target exposuretime.

In Equation (5), the difference between the target motion blur and theconverted motion blur is calculated by dividing the target motion blurby a converted motion vector. Next, in Equation (6), the target exposuretime is calculated by multiplying the current exposure time by thedivergence. The number of shutter speed steps to change from the currentexposure time in order to achieve the target exposure time calculatedthrough Equation (6) is calculated using Equation (7).

FIG. 16A illustrates a case where the current converted motion blur is40 pixels. Assuming the target motion blur is 5 pixels, DIFF is ⅛x; thusfor the current exposure time of 1/125 seconds, the target exposure timeis 1/1000 seconds, meaning that the number of steps the shutter speed ischanged is −3 steps.

FIG. 16B illustrates a case where the current converted motion blur is20 pixels. Assuming the target motion blur is 5 pixels, DIFF is ¼x; thusfor the current exposure time of 1/125 seconds, the target exposure timeis 1/500 seconds, meaning that the number of steps the shutter speed ischanged is −2 steps.

FIG. 16C illustrates a case where the current converted motion blur is10 pixels. Assuming the target motion blur is 5 pixels, DIFF is ½x; thusfor the current exposure time of 1/125 seconds, the target exposure timeis 1/250 seconds, meaning that the number of steps the shutter speed ischanged is −1 steps.

Returning to FIG. 15 , in step S1502, the motion blur notification planegeneration unit 1402 creates an image plane for notifying the user ofthe motion blur (a motion blur notification plane) on the basis of thedivergence between the converted motion blur and the target motion blurfor each pixel (the divergence for each pixel) calculated in step S1501.

Three examples of the motion blur notification image will be describedhere with reference to FIGS. 9A to 9E. Displaying the motion blurnotification image in the display unit 109 during the preparatoryshooting makes it possible for the user to easily confirm the motionblur.

FIGS. 9A to 9C illustrate an example of notifying the user of the motionblur by displaying an icon. A method for generating the motion blurnotification image by displaying an icon will be described here. In stepS1502, of the converted motion blur for each pixel, the motion blurnotification plane generation unit 1402 calculates the percentage of thenumber of pixels having a converted motion blur of a predetermined valueor higher, with respect to the entire screen. When the percentage isgreater than or equal to a predetermined percentage, the motion blurnotification plane generation unit 1402 determines the converted motionblur, from among the converted motion blur at greater than or equal to apredetermined value, that occupies the greatest percentage in thescreen.

The method for determining the converted motion blur that occupies thegreatest percentage of the screen will be described with reference toFIG. 10 . FIG. 10 is a diagram illustrating a histogram of convertedmotion blur having a predetermined value or higher in the sceneillustrated in FIG. 5A. The motion blur notification plane generationunit 1402 can generate a histogram such as that illustrated in FIG. 10 ,and then divide ranges of the converted motion blur on the basis of thegenerated histogram. The number of classes in the histogram can be setin advance. For example, consider a case where peaks (local maximumpoints) are present in the frequency of the histogram, as indicated inFIG. 10 . In this case, the motion blur notification plane generationunit 1402 can divide the range of the converted motion blur so that eachdivided range has one peak, with each peak serving as a representativemotion blur for a corresponding part of the object. Note that the peaksin the histogram can be detected using a method similar to that used fordetecting peaks in signal waveforms.

For example, three peaks 1001 to 1003 are present in the frequency ofthe histogram 1000 illustrated in FIG. 10 . As such, the motion blurnotification plane generation unit 1402 divides the range of theconverted motion blur at the positions where the frequency of theconverted motion blur first becomes a local minimum, from each peak. Themotion blur notification plane generation unit 1402 then assigns adisplay color to each divided range on the basis of the divergencescorresponding to the peaks 1001 to 1003. For example, the motion blurnotification plane generation unit 1402 assigns red to the divided rangeincluding the peak 1003, where three steps' (or more) worth of a shutterspeed change is required, as illustrated in FIG. 16A. Likewise, themotion blur notification plane generation unit 1402 assigns green to thedivided range including the peak 1002, where two steps' worth of ashutter speed change is required, as illustrated in FIG. 16B. Finally,the motion blur notification plane generation unit 1402 assigns blue tothe divided range including the peak 1001, where one step's worth of ashutter speed change is required, as illustrated in FIG. 16C.

When the display colors have been assigned in this manner, and thepercentage of the converted motion blur corresponding to the peak 1003is the highest as illustrated in FIG. 10 , the motion blur notificationplane generation unit 1402 creates the red motion blur notification icon901, as indicated in FIG. 9A, as the motion blur notification plane.Then, in step S406, the image superimposing unit 304 generates a motionblur notification image such as that illustrated in FIG. 9A bysuperimposing the motion blur notification plane, including the motionblur notification icon 901, onto the preparatory shooting image.

When the peak 1002 is higher than the peak 1003 and the percentage ofthe converted motion blur corresponding to the peak 1002 is the highest,the motion blur notification plane generation unit 1402 creates a greenmotion blur notification icon 902, as indicated in FIG. 9B, as themotion blur notification plane. Then, in step S406, the imagesuperimposing unit 304 generates a motion blur notification image suchas that illustrated in FIG. 9B by superimposing the motion blurnotification plane, including the motion blur notification icon 902,onto the preparatory shooting image.

Likewise, when the peak 1001 is higher than the peak 1003 and thepercentage of the converted motion blur corresponding to the peak 1001is the highest, the motion blur notification plane generation unit 1402creates a blue motion blur notification icon 903, as indicated in FIG.9C, as the motion blur notification plane. Then, in step S406, the imagesuperimposing unit 304 generates a motion blur notification image suchas that illustrated in FIG. 9C by superimposing the motion blurnotification plane, including the motion blur notification icon 903,onto the preparatory shooting image.

Although the foregoing describes dividing the converted motion blurranges using local minimum points in the histogram, the converted motionblur ranges may be divided at a predetermined threshold instead.Alternatively, a configuration in which the division points are variedin accordance with conditions may be employed as well. Even if such adivision method is employed, the motion blur notification planegeneration unit 1402 can create, as the motion blur notification plane,a motion blur notification icon having a display color corresponding tothe divided range including the converted motion blur that occupies thehighest percentage in the screen.

FIG. 9D illustrates an example of notifying the user of the motion blurby displaying frames. A method for generating the motion blurnotification image by displaying frames will be described here. In stepS1502, of the pixels within divided regions of the shooting screen, themotion blur notification plane generation unit 1402 calculates thepercentage of the number of pixels having a converted motion blur of apredetermined value or higher, with respect to the overall dividedregions. For divided regions in which that percentage is greater than orequal to a predetermined percentage, the motion blur notification planegeneration unit 1402 creates motion blur notification frames, asillustrated in FIG. 9D, as the motion blur notification plane. At thistime, the motion blur notification plane generation unit 1402 determinesthe display color of the motion blur notification frames correspondingto each divided region in accordance with the divergence in each dividedregion.

For example, the motion blur notification plane generation unit 1402assigns red to divided regions having a divergence that corresponds to achange of three steps (or more) in the shutter speed. Likewise, themotion blur notification plane generation unit 1402 assigns green todivided regions having a divergence that corresponds to a change of twosteps in the shutter speed. Finally, the motion blur notification planegeneration unit 1402 assigns blue to divided regions having a divergencethat corresponds to a change of one step in the shutter speed. In thiscase, the motion blur notification plane generation unit 1402 createsthe red motion blur notification frames 904 for the parts correspondingto the legs, where there is a high amount of converted motion blur andthus a higher divergence, and creates the green motion blur notificationframes 905 for parts corresponding to the head and tail, where there isa medium amount of converted motion blur and thus a medium divergence.The motion blur notification plane generation unit 1402 also createsblue motion blur notification frames 906 for parts corresponding to thetrunk, where there is little converted motion blur and thus littledivergence. Then, in step S406, the image superimposing unit 304generates a motion blur notification image such as that illustrated inFIG. 9D by superimposing the motion blur notification plane, includingthe motion blur notification frames 904 to 906, onto the preparatoryshooting image.

FIG. 9E illustrates an example of notifying the user of the motion blurby displaying edges in which motion blur has arisen in an enhancedmanner. A method for generating the motion blur notification image bydisplaying the edges in which motion blur has arisen in an enhancedmanner will be described here. In step S1502, the motion blurnotification plane generation unit 1402 detects the edge strength of thepreparatory shooting image. It is assumed that the edge strength iscalculated using a known technique such as a Sobel filter, so this willnot be described here. The motion blur notification plane generationunit 1402 then extracts pixels for which the edge strength is greaterthan or equal to a predetermined value and for which the convertedmotion blur is greater than or equal to a predetermined value. Themotion blur notification plane generation unit 1402 then creates amotion blur notification plane, in which the edge areas where motionblur has arisen are displayed in an enhanced manner for the extractedpixels, as indicated by motion blur notification edges 907 to 909 inFIG. 9C. At this time, the motion blur notification plane generationunit 1402 determines the display color of each pixel in the motion blurnotification edges in accordance with the divergence in those pixels.

For example, the motion blur notification plane generation unit 1402assigns red to pixels having a divergence that corresponds to a changeof three steps (or more) in the shutter speed. Likewise, the motion blurnotification plane generation unit 1402 assigns green to pixels having adivergence that corresponds to a change of two steps in the shutterspeed. Finally, the motion blur notification plane generation unit 1402assigns blue to pixels having a divergence that corresponds to a changeof one step in the shutter speed. In this case, the motion blurnotification plane generation unit 1402 creates the red motion blurnotification edges 907 for pixels in the parts corresponding to thelegs, where there is a high amount of converted motion blur and thus ahigher divergence, and creates the green motion blur notification edges908 for pixels in the parts corresponding to the head and tail, wherethere is a medium amount of converted motion blur and thus a mediumdivergence. The motion blur notification plane generation unit 1402 alsocreates blue motion blur notification edges 909 for the pixels in theparts corresponding to the trunk, where there is little converted motionblur and thus little divergence. Then, in step S406, the imagesuperimposing unit 304 generates a motion blur notification image suchas that illustrated in FIG. 9E by superimposing the motion blurnotification plane, including the motion blur notification edges 907 to909, onto the preparatory shooting image.

Although FIG. 9E illustrates an example in which the edges are displayedin an enhanced manner by varying the display colors of the motion blurnotification edges in accordance with the divergence, the enhanceddisplay method is not limited thereto. A method of displaying the edgesin an enhanced manner by varying the thicknesses of the motion blurnotification edges in accordance with the divergence can be given asanother example of an enhanced display method.

According to the third embodiment as described thus far, the imagecapturing apparatus 100 varies the form of the motion blur notificationin accordance with the divergence between the motion blur amount thatwill arise in the actual shooting, and the target motion blur amount.This makes it possible for the photographer to easily know, in advance,how much motion blur will arise in the actual shooting.

Although the present embodiment describes a configuration in which themotion blur notification image is displayed in the display unit 109 asthe method for notifying the user of motion blur, the method fornotifying the user of motion blur is not limited thereto. For example, aconfiguration in which the user is notified of motion blur by outputtingsound may be employed. In this case, for example, the control unit 101may output a motion blur notification sound from a speaker (not shown)when, of the converted motion blur for each pixel, the percentage of thenumber of pixels having a converted motion blur of a predetermined valueor higher, with respect to the entire screen, is greater than or equalto a predetermined percentage. In this case, the control unit 101 maydivide the range of the converted motion blur using the same method asthat described with reference to FIG. 10 , and assign a different volumeto each divided range depending on the divergence. The control unit 101may then output the motion blur notification sound at a volumecorresponding to the divided range including the converted motion blurthat occupies the highest percentage in the screen. Alternatively, thecontrol unit 101 may assign a different type of sound to each dividedrange depending on the divergence, and output the type of motion blurnotification sound corresponding to the divided range including theconverted motion blur that occupies the highest percentage in thescreen. In other words, the control unit 101 can change at least one ofthe type of the sound and the volume in accordance with the divergence.

Additionally, although the present embodiment describes, as a method ofmaking the motion blur notification, an example in which the displaycolor of the motion blur notification icon is changed in accordance withthe divergence, the item to be varied in accordance with the divergenceis not limited to the display color. For example, a configuration may beemployed in which the size of the motion blur notification icon isvaried in accordance with the divergence. For example, a configurationmay be employed in which the icon size is increased when the divergenceis higher, and the icon size is reduced when the divergence is lower.Varying the icon size in accordance with the divergence in this mannermakes it possible for the user to easily confirm the intensity of themotion blur.

Accordingly, the notification image generating unit 1400 can carry outcontrol for displaying blur notification icons, which have differentappearances depending on the divergence, along with the preparatoryshooting image. For example, the notification image generating unit 1400can vary at least one of the color and size of the blur notificationicon in accordance with the divergence.

Additionally, although the present embodiment describes, as a method ofmaking the motion blur notification, an example in which the displaycolor of the motion blur notification frames or the motion blurnotification edges is changed in accordance with the divergence, theitem to be varied in accordance with the divergence is not limited tothe display color. For example, a configuration may be employed in whichthe thickness of the lines of the motion blur notification frames or themotion blur notification edges is varied in accordance with thedivergence. For example, a configuration may be employed in which theframes or edges are displayed thicker when there is a higher divergence,and thinner when there is a lower divergence. Varying the linethicknesses in accordance with the divergence in this manner makes itpossible for the user to easily confirm the intensity of the motionblur.

Accordingly, the notification image generating unit 1400 can carry outcontrol for displaying a plurality of blur notification frames, whichsurround a plurality of divided regions in the preparatory shootingimage where motion blur is arising, along with the preparatory shootingimage. The notification image generating unit 1400 can then vary theappearance of each blur notification frame in accordance with thedivergence in the divided region corresponding to the blur notificationframe. For example, the notification image generating unit 1400 can varyat least one of the color and thickness of each blur notification framein accordance with the divergence in the divided region corresponding tothe blur notification frame. Note that the region where the blurnotification frame is displayed is not limited to the divided region,and a motion blur notification can be made using a blur notificationframe for any desired partial region in the preparatory shooting imagewhere motion blur is arising. Additionally, the notification imagegenerating unit 1400 can display a plurality of edge regions wheremotion blur is arising in the preparatory shooting image, in an enhancedmanner with different forms depending on the divergence in each of theedge regions. The “enhanced display” includes at least one of displayingthe plurality of edge regions with different colors depending on thedivergence in each of the edge regions, and displaying the plurality ofedge regions using lines of different thicknesses depending on thedivergence in each of the edge regions.

Fourth Embodiment

Although the third embodiment described a configuration in which amotion amount detected from a preparatory shooting image is convertedinto a motion blur amount arising in actual shooting, the fourthembodiment will describe a configuration in which a motion amount of animage capturing apparatus that is carrying out preparatory shooting isconverted into a motion blur amount arising in actual shooting. Althoughthe fourth embodiment will be described with reference to an imagecapturing apparatus 1700, illustrated in FIG. 17 , instead of the imagecapturing apparatus 100 illustrated in FIG. 1 , the configuration andoperations of the image capturing apparatus 1700 share some common partswith the image capturing apparatus 100 according to the thirdembodiment. The following will primarily describe points that aredifferent from the third embodiment.

The image capturing apparatus 1700 illustrated in FIG. 17 differs fromthe image capturing apparatus 100 illustrated in FIG. 1 in that theangular velocity detection unit 1101 has been added, and the imageprocessing unit 107 has been replaced with an image processing unit1701.

The angular velocity detection unit 1101 is an angular velocity sensor,for example, which detects the angular velocity of the image capturingapparatus 1700 itself, caused by the user's hand shaking, the user'scamerawork, or the like, in the yaw direction and the pitch direction.Any desired known method can be used as the method by which the angularvelocity detection unit 1101 detects the angular velocity.

The image processing unit 1701 includes a notification image generatingunit 1800 (described later), which generates a motion blur notificationimage by superimposing an image plane enabling motion blur to be easilyconfirmed over an image stored in the RAM 103.

An example of the configuration of the notification image generatingunit 1800 included in the image processing unit 1701 will be describednext with reference to FIG. 18 . The notification image generating unit1800 illustrated in FIG. 18 differs from the notification imagegenerating unit 1400 illustrated in FIG. 14 in that the motion vectorcalculation unit 301 has been replaced with the image motion informationconversion unit 1201. The operations of the notification imagegenerating unit 1800 will be described in detail later with reference toFIG. 19 .

Next, the process by which the notification image generating unit 1800generates the motion blur notification image (step S204 in FIG. 2 ) willbe described in detail with reference to FIG. 19 . In FIG. 19 , theprocesses of steps S401, S403, S404, and S406 are the same as theprocesses described in the first embodiment with reference to FIG. 4 .Additionally, the process of step S1301 is the same as the processdescribed in the second embodiment with reference to FIG. 13 . Theprocess of step S1501 is the same as the process described in the thirdembodiment with reference to FIG. 15 .

In step S1901, the motion blur notification plane generation unit 1402creates an image plane for notifying the user of the motion blur (themotion blur notification plane) on the basis of the converted motionblur calculated in step S404. The process of step S1901 is the same asin the third embodiment (step S1502 in FIG. 15 ) in that the motion blurnotification plane is created with different forms depending on thedivergence between the converted motion blur and the target motion blur.However, in the fourth embodiment, the motion vectors are uniform forall of the pixels, and thus the divergence between the converted motionblur and the target motion blur is also uniform for all of the pixels,which means that a histogram such as that illustrated in FIG. 10 cannotbe generated. Accordingly, the motion blur notification plane generationunit 1402 determines the form of the motion blur notification plane by,for example, comparing the divergence that is uniform for all of thepixels to one or more thresholds. For example, assume that D representsthe divergence, and T1, T2, and T3 represent three thresholds(T1<T2<T3). If T1≤D<T2, the motion blur notification plane generationunit 1402 creates the blue motion blur notification icon 903 (FIG. 9C)as the motion blur notification plane. If T2≤D<T3, the motion blurnotification plane generation unit 1402 creates the green motion blurnotification icon 902 (FIG. 9B) as the motion blur notification plane.If T3≤D, the motion blur notification plane generation unit 1402 createsthe red motion blur notification icon 901 (FIG. 9A) as the motion blurnotification plane.

According to the fourth embodiment as described thus far, the imagecapturing apparatus 1700 converts the motion amount of the imagecapturing apparatus 1700 carrying out preparatory shooting into themotion blur amount that will arise in the actual shooting. This makes itpossible to obtain the converted motion blur with a comparatively lightprocessing load.

Fifth Embodiment

According to Japanese Patent Laid-Open No. 2008-172667, detecting aregion of motion between images captured in time series duringpreparatory shooting, and emphasizing that region of motion, makes iteasy to prompt the user to reduce blur.

However, Japanese Patent Laid-Open No. 2008-172667 does not take intoaccount shooting an image at an appropriate framerate, shutter speed,and so on in accordance with the speed, movement amount, and so on ofthe object, as a way to extract the region of motion between images intime series. For example, detecting motion accurately requires that themoving object be shot without cumulative blur, and to accomplish this,it is necessary to increase the shutter speed. Furthermore, for objectsmoving at high speeds, the range in which the moving object can bedetected is limited as well, and it is therefore necessary to increasethe framerate so as to reduce the amount of movement between images.However, carrying out preparatory shooting at a high framerate reducesthe quality of the preparatory shooting image, and also consumes morepower.

In light of such circumstances, the present embodiment will describe theconfiguration for striking a desirable balance between the accuracy ofmotion detection and improvement in the quality of the preparatoryshooting image.

In the fifth embodiment, the basic configuration of the image capturingapparatus 100 is the same as in the first embodiment (see FIG. 1 ). Thefollowing will primarily describe points that are different from thefirst embodiment.

A shooting process executed by the image capturing apparatus 100 will bedescribed with reference to FIG. 20 . Unless otherwise specified, theprocesses in the respective steps of this flowchart are realized by thecontrol unit 101 executing the aforementioned control programs. Theprocessing illustrated in this flowchart starts when the user turns theimage capturing apparatus 100 on and an operating mode of the imagecapturing apparatus 100 enters a shooting mode.

In step S2008, the control unit 101 determines shooting conditions forthe preparatory shooting in order to detect the motion amount of anobject in the composition. Specifically, the control unit 101 calculatesshooting conditions suited to the detection of the motion amount of theobject in the composition on the basis of a relationship between themotion amount of the object detected from a preparatory shooting imageshot under initial shooting conditions, and a target blur amount (targetmotion blur) at which a motion blur notification is to be made. Theprocess of step S2008 will be described in detail later with referenceto FIG. 21 .

Next, the process by which the control unit 101 determines the shootingconditions for preparatory shooting (step S2008 in FIG. 20 ) will bedescribed in detail with reference to FIG. 21 .

In step S2101, the control unit 101 sets the initial shooting conditionsin the image capturing unit 105, and captures the preparatory shootingimage in a continuous manner. Under the initial shooting conditions, theframerate and the shutter speed are set to the highest (fastest) valueswithin a range that does not affect processing for calculatingevaluation values used in the auto function control carried out by atypical camera, such as automatic exposure (AE) and autofocus (AF)control. An appropriate aperture value and ISO sensitivity are also setso that the image can be shot under appropriate exposure conditionsusing the fast shutter speed which has been set. Accordingly, a movingobject in the preparatory shooting image captured under the initialshooting conditions has a comparatively low amount of cumulative blur.There is also a comparatively low amount of movement in the objectbetween the consecutive images. As such, the motion amount can bedetected with a high level of accuracy, even for an object moving athigh speed. However, the ISO sensitivity is set to a comparatively highvalue in order to achieve the appropriate exposure conditions, whichmeans that the preparatory shooting image contains a comparatively largeamount of noise.

In step S2102, the notification image generating unit 300 calculatesmotion vectors between the preparatory shooting images shot under theinitial shooting conditions, under the control of the control unit 101,so as to calculate a motion amount of the object (the magnitude of themotion vectors). The method for calculating the motion vectors is thesame as the calculation method described in the first embodiment withreference to step S402 in FIG. 4 .

In step S2103, the control unit 101 specifies a relationship between themotion amount of the object, calculated in step S2102, and the targetmotion blur. The “target motion blur” is a permissible motion amount atwhich the motion amount of the object in the captured image does notproduce motion blur that stands out. The permissible motion amount isdetermined on the basis of the size and the number of pixels of theimage sensor, which is a CCD, a CMOS sensor, or the like, and theresolution of the display in which the shot image is displayed. Forexample, if the size of the image sensor is APS-C and the number ofpixels is 200,000 pixels, and the resolution of the display is full HD(1920×1080 pixels), the permissible motion amount is assumed to be 5pixels or less. In this case, adjusting the framerate so that motionless than or equal to 5 pixels can be detected between images makes itpossible to detect the motion of the object with an appropriate level ofaccuracy.

A specific example of the relationship between the motion amount of theobject and the target motion blur will be described with reference toFIGS. 22A and 22B. FIGS. 22A and 22B express the relationship betweenthe motion amount of the object, calculated from the preparatoryshooting image, and the target motion blur (permissible motion amount),using the magnitude (number of pixels) of motion vectors. FIG. 22Aillustrates a state in which the relationship between the motion amountof the object and the target motion blur is not appropriate, whereasFIG. 22B illustrates a state in which the relationship between themotion amount of the object and the target motion blur is appropriate.

The state illustrated in FIG. 22A is a state in which image capturingcontrol for the preparatory shooting image is carried out using aframerate of 60 fps as the initial shooting conditions. At this time, amotion vector 2201 of the object is 10 pixels, and a motion vector 2202corresponding to the permissible motion amount is 5 pixels, which meansthat the motion vector 2201 of the object is larger than the motionvector 2202 corresponding to the permissible motion amount. In thiscase, despite the motion vector 2202 corresponding to the permissiblemotion amount being 5 pixels, the motion amount of the object can onlybe detected in units of 10 pixels. This means that there is too muchmovement in the object for the framerate of 60 fps in the preparatoryshooting. In particular, when the object is soft or is undergoingrotational movement, if the motion amount for detection is too coarse(the unit is too great), it becomes difficult to detect the correctmotion, which in turn makes it difficult to make a correct motion blurnotification and the process for notifying the user of motion blur forthe actual shooting (described later).

The state illustrated in FIG. 22B is a state in which image capturingcontrol for the preparatory shooting image is carried out using aframerate of 120 fps as the shooting conditions. At this time, a motionvector 2203 of the object is 5 pixels, and a motion vector 2204corresponding to the permissible motion amount is 5 pixels, which meansthat the motion vector 2203 of the object indicates the same movementamount as the motion vector 2204 corresponding to the permissible motionamount. In other words, it can be seen that the motion amount can bedetected at the appropriate unit (that is, units of 5 pixels) at 120fps. Thus if the motion vectors can be detected at units less than orequal to the permissible motion amount in this manner, whether or notthe motion blur in the actual shooting will exceed the permissiblemotion amount can be determined with a high level of accuracy.

Expressing the conditions for detecting motion with a high level ofaccuracy as a relational expression between the motion blur of theobject and the target motion amount results in the following Equation(8).

n=magnitude of motion vector/permissible motion amount  (8)

Here, n being greater than 1 indicates that the unit at which the motionamount can be detected is too high, whereas n being less than or equalto 1 indicates that the motion amount can be detected at the appropriateunit.

In addition to a method in which the permissible motion amount is set inadvance in consideration of the accuracy of the detection of the motionblur during the actual shooting, the permissible motion amount can alsobe determined in response to user instructions. Alternatively, thepermissible motion amount can be determined in accordance with theamount of motion blur after converting the length of the detected motionvectors into a magnitude of motion vectors corresponding to the shutterspeed for the actual shooting, which will be described later.

Returning to FIG. 21 , in step S2104, the control unit 101 calculatesthe shooting conditions at which the motion amount can be detected atthe appropriate unit, on the basis of the relationship between themotion amount of the object and the target motion blur specified in stepS2103; then, the shooting conditions for the preparatory shooting arechanged to the calculated shooting conditions. The control unit 101 thencontrols the image capturing unit 105 to carry out the preparatoryshooting under the post-change shooting conditions.

Note that the appropriate shooting conditions for detecting motion arenot necessarily achieved simply by increasing the framerate. If theframerate is too high, the exposure time will be shortened, resulting inimages that contain a high amount of noise, particularly for darklocations; in this case, it is possible that adverse effects will arisewith respect to detecting motion, such as the inability to accuratelydetermine the signals from the object. As such, the “appropriateshooting conditions for detecting motion” correspond to an exposure thatis as long as possible, within an exposure time at which the motionamount can be detected at a unit less than or equal to the permissiblemotion amount, and at which that motion amount can be detected.

Accordingly, the control unit 101 calculates the framerate and theshutter speed, which are the primary factors in the shooting conditions,using Equations (9) and (10).

framerate(fps)=initial framerate×n  (9)

shutter speed(s)=initial shutter speed×1/n  (10)

Here, n is a value determined through the aforementioned Equation (8).

The framerate and shutter speed calculated in this manner are importantshooting conditions for detecting movement. Additionally, to capture animage of the appropriate brightness, the control unit 101 also carriesout control for changing the aperture value and the ISO sensitivity inaccordance with the changes made to the framerate and the shutter speed,so that the exposure value does not change.

According to Equations (9) and (10), when n is less than 1, as in theexamples illustrated in FIGS. 22A and 22B, the framerate and shutterspeed both increase. Accordingly, the motion amount can be detected at aunit necessary for detecting the motion with a high level of accuracy.Additionally, according to Equations (9) and (10), when n is less than1, the framerate and shutter speed both decrease. Accordingly,situations where the framerate and shutter speed are increased more thanis necessary, resulting in a drop in the quality of the preparatoryshooting image, can be suppressed.

Note that there are cases where the framerates that can be set by thecontrol unit 101 have discrete values. Thus, to be more exact, thecontrol unit 101 does not need to set the initial framerate to strictlyn times, but rather may set the initial framerate to approximately ntimes (approximately (motion amount)/(target motion blur amount) times).The range indicated by “approximately” is determined as appropriate inaccordance with the framerate values that can be set by the control unit101. The same applies to the shutter speed, and thus the control unit101 may set the initial shutter speed to approximately 1/n times.

Furthermore, the control unit 101 may change the initial framerate to avalue higher (or lower) that approximately n times. Even in this case,changing the framerate of the preparatory shooting on the basis of adifference between the magnitude of the motion vectors (the motionamount) and the permissible motion amount (the target motion bluramount) makes it possible to achieve the desired balance betweenimproving the accuracy of the motion detection and improving the qualityof the preparatory shooting image.

FIGS. 23A and 23B are timing charts illustrating a process by which thecontrol unit 101 determines shooting conditions for preparatory shooting(step S2008 in FIG. 20 ). FIGS. 23A and 23B indicate the timings of thepreparatory shooting and the actual shooting. FIG. 23A illustrates thetimings of the preparatory shooting and the actual shooting when theprocess of step S2008 is not carried out, whereas FIG. 23B illustratesthe timings of the preparatory shooting and the actual shooting when theprocess of step S2008 is carried out. Reference signs 2301 to 2318indicate pairs of sequential and adjacent images necessary for detectingmotion.

In the case of FIG. 23A (when the process of step S2008 is not carriedout), the control unit 101 carries out preparatory shooting continuouslyat a fixed, high framerate, for example. In other words, the controlunit 101 continuously detects the motion amount of the object betweenthe adjacent preparatory shooting images shot under the initial shootingconditions (reference signs 2301 to 2312), until the actual shooting iscarried out. Upon detecting that the user has depressed the shutterbutton, the control unit 101 carries out the actual shooting (referencesign 2321).

On the other hand, in the case illustrated in FIG. 23B (when the processof step S2008 is carried out), the preparatory shooting is carried outunder the initial shooting conditions until the timings indicated byreference signs 2313 to 2316, in the same manner as the timingsindicated by the reference signs 2301 to 2304. However, due to theprocess of step S2008, the shooting conditions for the preparatoryshooting are changed on the basis of the relationship between the motionamount of the object and the target motion blur, and thus thepreparatory shooting is carried out under the post-change shootingconditions at the timings indicated by reference signs 2317 and 2318.Although FIG. 22B illustrates an example in which the frame rateincreases, FIG. 23B illustrates an example in which the frameratedecreases. The actual shooting (reference sign 2322) is carried outthereafter.

The processes following step S2008 are the same as those described inthe first embodiment (FIG. 2 ). However, in the process for convertingthe motion vectors from the preparatory shooting into the motion blurthat will arise in the actual shooting (step S404 of FIG. 4 , which is apart of step S204), the motion vectors from the preparatory shooting aremotion vectors corresponding to the shooting conditions changed by theprocess of step S2008. As such, when, for example, the change has beenmade as indicated in FIG. 22B, the conversion is carried out asindicated in FIG. 24 (instead of FIG. 8 ). If it is determined in stepS206 that the shutter button has not been pressed, the process returnsto step S2008. In this case, in step S2101 of FIG. 21 , the shootingconditions that have been changed as a result of the processing in theprevious step S2008 are used instead of the initial shooting conditions.

Note that with respect to the determination (change) of the shootingconditions in step S2008, the control unit 101 may be configured tochange the conditions upon detecting that the motion of the object is ina stable state. Additionally, the control unit 101 may, on the basis ofchanges in the motion vectors, determine whether the motion of theobject is in a stable state after the shooting conditions for thepreparatory shooting have been changed once, and may then change theshooting conditions for the preparatory shooting again if the amount ofchange in the motion vectors per unit of time has increased. In otherwords, if a change has occurred in the motion amount detected from thepreparatory shooting image after the shooting conditions such as theframerate have been changed, the control unit 101 may change theshooting conditions such as the framerate again.

According to the fifth embodiment as described above, the imagecapturing apparatus 100 changes the framerate for the preparatoryshooting on the basis of a difference between the motion amount detectedfrom the preparatory shooting image, and the target motion blur amount.This makes it possible to achieve the desired balance between improvingthe accuracy of the motion detection and improving the quality of thepreparatory shooting image.

Note that by combining the configuration of the third embodiment withthe configuration of the present embodiment, the user can be notified ofmotion blur using three colors (red, green, and blue) depending on theamount of converted motion blur. In this case, the control unit 101 setsthe permissible motion amount so that the narrowest range of the motionamount can be detected, so that it is possible to express the range ofthe motion amount represented by each color. For example, if the colorsrepresenting the motion blur relative to the magnitude of the motionvectors are 0 to 10 pixels for blue, 11 to 15 pixels for green, and 16pixels or more for red, the range for green is the narrowest section asa motion amount. As such, the control unit 101 sets 5 pixels, i.e., therange for green (11 to 15), as the permissible motion amount.

Although the present embodiment describes an example in which theframerate and the shutter speed are changed as the shooting conditionsfor the preparatory shooting in order to detect motion, it is alsopossible to change only one of these conditions.

Additionally, a configuration may be employed in which the permissiblemotion amount is set as the magnitude of motion vectors within a rangethat does not exceed a limit range for motion detection, in a situationwhere the user has not intentionally determined the permissible motionamount. At this time, it is necessary to determine whether the motion ofthe object has exceeded the range for motion detection. Using an SADvalue during motion detection to detect whether an amount of change hasdropped below a predetermined value, measuring motion of the imagecapturing apparatus 100 itself using gyro information of the like, andso on can be used as methods to make this determination. If it isdetermined that the permissible motion amount has been exceeded, thecontrol unit 101 carries out control for increasing the shutter speed,the framerate, or the like.

Furthermore, if the object is moving quickly and the framerate for thepreparatory shooting cannot be changed, the control unit 101 may reducethe shutter speed only, within a range that does not lead to a drop inthe framerate.

OTHER EMBODIMENTS

Embodiment(s) of the present invention can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment(s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s). Thecomputer may comprise one or more processors (e.g., central processingunit (CPU), micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM), a read only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2018-248373, filed Dec. 28, 2018 which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. A notifying apparatus comprising at least oneprocessor and/or at least one circuit which functions as: a detectingunit configured to detect, in a predetermined period, a motion amount ofan image capturing apparatus carrying out first shooting that is carriedout repeatedly at predetermined intervals of time; a converting unitconfigured to convert the motion amount into a motion blur amount thatwill arise in second shooting, on the basis of the predetermined periodand an exposure time used in the second shooting; and a notifying unitconfigured to make a notification of motion blur on the basis of themotion blur amount, wherein the notifying unit changes a form of thenotification in accordance with a magnitude of the motion blur amount.2. The notifying apparatus according to claim 1, wherein the at leastone processor and/or at least one circuit is further configured tofunction as: a display unit configured to display the image obtainedfrom the first shooting, wherein the notifying unit controls the displayunit to display an icon having a different appearance depending on themagnitude of the motion blur amount along with the image.
 3. Thenotifying apparatus according to claim 2, wherein the notifying unitchanges at least one of a color and size of the icon in accordance withthe magnitude of the motion blur amount.
 4. The notifying apparatusaccording to claim 1, wherein the at least one processor and/or at leastone circuit is further configured to function as: a display unitconfigured to display the image obtained from the first shooting,wherein the notifying unit controls the display unit to display aplurality of frames surrounding a plurality of partial regions wheremotion blur is arising in the image, along with the image, and changesan appearance of each frame in accordance with the magnitude of themotion blur amount in the partial region corresponding to that frame. 5.The notifying apparatus according to claim 4, wherein the notifying unitchanges at least one of a color and a thickness of each frame inaccordance with the magnitude of the motion blur amount in the partialregion corresponding to that frame.
 6. The notifying apparatus accordingto claim 1, wherein the at least one processor and/or at least onecircuit is further configured to function as: a display unit configuredto display the image obtained from the first shooting, wherein thenotifying unit controls the display unit to display a plurality of edgeregions where motion blur is arising in the image in an enhanced mannerwith different appearances depending on the magnitude of the motion bluramount in each of those edge regions.
 7. The notifying apparatusaccording to claim 6, wherein the enhanced display includes at least oneof displaying the plurality of edge regions with different colorsdepending on the magnitude of the motion blur amount in each of thoseedge regions, and displaying the plurality of edge regions with lines ofdifferent thicknesses depending on the magnitude of the motion bluramount in each of those edge regions.
 8. An image capturing apparatuscomprising: the notifying apparatus according to claim 1; and an imagesensor.
 9. A notifying method executed by a notifying apparatus,comprising: detecting, in a predetermined period, a motion amount of animage capturing apparatus carrying out first shooting that is carriedout repeatedly at predetermined intervals of time; converting the motionamount into a motion blur amount that will arise in second shooting, onthe basis of the predetermined period and an exposure time used in thesecond shooting; and making a notification of motion blur on the basisof the motion blur amount, wherein a form of the notification is changedin accordance with a magnitude of the motion blur amount.
 10. Anon-transitory computer-readable storage medium which stores a programfor causing a computer to execute a notifying method comprising:detecting, in a predetermined period, a motion amount of an imagecapturing apparatus carrying out first shooting that is carried outrepeatedly at predetermined intervals of time; converting the motionamount into a motion blur amount that will arise in second shooting, onthe basis of the predetermined period and an exposure time used in thesecond shooting; and making a notification of motion blur on the basisof the motion blur amount, wherein a form of the notification is changedin accordance with a magnitude of the motion blur amount.